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Field of the Invention 
15 This invention relates generally to imaging systems and methods and, more 

particularly, to thermal imaging systems and methods. 

Copending Applications 

This application is copending with U.S. Patent Application Ser. No. 



atty. docket number BOEI-1-1 180, and U.S. Patent Application Ser. No. , atty. 

20 docket number BOEI-1-1 182, all filed and all are hereby incorporated by 

reference. 

Background of the Invention 

Airborne and satellite remote thermal sensing of the ground has been widely used for 
various change detection applications, such as agriculture. The remote sensors detect 
25 differences in temperature across the surface of the earth. The sensed temperatures are then 
converted into images used by the change detection applications. FIGURE 1 illustrates an 
example image generated by prior art remote sensing systems. The image in FIGURE 1 is a 
resized or zoomed-in view produced by a Land Remote Sensing Satellite Program 
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(LANDSAT-7) satellite operated by NASA. The thermal images produced by prior known 



methods are useful primarily in low resolution applications. 

It would be desirable to provide thermal images for high resolution applications. 
However, there exists an unmet need for generating remotely sensed thermal images at 
5 higher resolutions to provide more accurate interpretation of thermal data. 

Summary of the Invention 

The present invention provides a system, method, and computer program for 
generating an image based on thermal data. Thermal data images produced by the present 
invention include a high degree of radiometric accuracy for use in various thermal image 
10 analysis applications. 



data including units of data in a first resolution is received and converted into a radiance 
value. The radiance value pan band data is converted into a power value. An array of thermal 
band data that has units of data in a second resolution is received. The received thermal band 

15 data is converted into a temperature value and the spatial resolution of the temperature value 
thermal band data is changed to match the spatial resolution of the received pan band data. A 
color array is generated based on the received array of the pan band data and the array of the 
changed thermal band data. The color array includes units having red, green, and blue values. 
An image is generated based on the generated color array. 

20 In one aspect of the invention, a red value for each unit is generated by adding a value 

of each corresponding unit in the pan band data to a value of each corresponding unit in the 
thermal band data. Green and blue values for each unit are generated by determining a mean 
or average temperature value for the received thermal band data and adding a value of each 
corresponding unit in the pan band data to the determined mean temperature value. 

25 In another aspect of the invention, the generated image is displayed on a display 

device or a hard copy device produced on a printing device. 

Brief Description of the Drawings 

The preferred and alternative embodiments of the present invention are described in 
detail below with reference to the following drawings. 
30 FIGURE 1 is a thermal image produced by a prior art method; 

FIGURE 2 is a block diagram of an example system formed in accordance with the 
present invention; 



FIGURE 3 is a flow diagram of an example process performed by the system shown 
in FIGURE 2; 



According to exemplary embodiments of the present invention, an array of pan band 
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FIGURE 4 illustrates how data from a thermal band is converted into color; and 
FIGURE 5 is a thermal image produced by the process shown in FIGURE 3. 

Detailed Description of the Invention 

The present invention is a system, method, and computer program for generating 
5 thermal images from data received from a remote earth sensor system. FIGURE 2 illustrates 
an exemplary system 40 for generating thermal images. The system 40 includes a sensor 
system 44, a processor 46, one or more display devices 48 or printing devices, and 
memory 49. The sensor system 44 provides panchromatic (pan) band data, which is 
electromagnetic radiation in the visible spectrum (0.4-0.7 micrometers), and thermal infrared 
10 (IR) band data, which is electromagnetic radiation having wavelengths between 3 and 25 
micrometers. The processor 46 is coupled to the display device 48 and the memory 49. The 
processor 46 receives and processes the panchromatic band and thermal band data to produce 
an image for display on the display device 48. 

In one embodiment the sensor system 44 is a Land Remote Sensing Satellite Program 
15 satellite (LANDS AT) system that generates multiple bands of data wherein each band is a 
collection of radiation from different ranges of the electromagnetic spectrum. Other sensor 
systems may be used. 

FIGURE 3 illustrates an exemplary process 100 that is performed by the processor 46 

in FIGURE 2 for generating a thermal image for display on the display device 48. At a 

20 block 104, the processor 46 receives thermal band data from the sensor system 44. The 

thermal band includes units with associated data that are formatted in digital number format. 

At a block 106, each of the digital number formatted units of data is converted into a 

temperature value. Temperature conversion is done using equations supplied by the sensor 

system operator or manufacturer, such as NASA. In one non-limiting example, the data in 

25 digital number format is converted to a temperature value in the Kelvin scale, see below. 
(K2) 



T = 



where 

T = Effective at-satellite temperature in Kelvin 
K2 = Calibration constant 2 in Kelvin 
30 Kl = Calibration constant 1 in watts/(meter * sr * micron) 

L = spectral radiance in watts^meter 2 * sr * micron) 
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At a block 108, the converted thermal band data are resized to match the same spatial 
resolution as that of the pan band data produced by the sensor system 44. If, for example, the 
sensor system 44 is a LANDS AT series system, the pan band is at a 15-meter resolution level 
and the thermal band data is at a 60-meter resolution level. In order to resize the thermal band 
5 data, each unit of data is split into eight subunits. Each subunit suitably has the same value as 
the original unit, thereby producing data at a 1 5-meter resolution level. 

The processor 46 receives pan band data in digital number format at a block 1 10. At a 
block 1 12, each unit of the pan band data is converted into a radiance value. EQUATION (1) 
below illustrates an exemplary formula for converting the digitally formatted information 
10 into a radiance value. 

{LMAX T - LMAXj) , _ , . * 

T = y- x (qcal - qcal mm) + LMIN T (1) 

(qcal max- qcal min) 

where 

15 Lt = spectral radiance in watts/(meter 2 * sr * micron) 

QCAL = quantized calibrated frame unit value in digital number (DN) format 
QCALMIN = minimized quantized calibrated frame unit value in DN format 
QCALMAX = maximum quantized calibrated frame unit value in DN format 
. LMINt = spectral radiance that is scaled to QCALMIN in watts^meter 2 * sr * 

20 micron) 

LMAXj = spectral radiance that is scaled to QCALMIN in watts/(meter 2 * sr * 
micron) 

At a block 1 14, each radiance value (R) is converted into a power value (P) according 

25 to EQUATION (2). 

P = R.(w u -w l ) (2) 

EQUATION (2) above generates a power value for each unit based on corresponding 
a radiance value with an upper width limit w u and a lower width limit wi of the pan band. The 
upper and lower limits w u and wi of the pan band are adjusted based on test data. Using the 
30 spectral response of bands in the sensor system 44, an equivalent uniform response band 
width with upper limit w u and lower limit wj is calculated using normal practice of using 
equal relative response point left and right with a uniform amplitude equal to the peak of the 
measured relative response. Other similar operations can be used to determine and adjust 
power. 
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At a block 1 16, a red value is generated by adding the corresponding pan band power 
value, as determined in the block 1 14, to the corresponding temperature value, as determined 
in the block 108, for each unit. At a block 120, a mean or average temperature value is 
determined for the data of the resized units of the thermal band data. It will be appreciated 
5 that other resizing techniques can be used. At a block 122, blue and green values are 
generated by adding the corresponding pan band power value to the determined mean or 
average temperature value. The generated red, green, and blue pixel values are stored for 
later use. At a decision block 124, the process 100 determines whether the red, green, and 
blue values for all the units have been generated. If not all the pixel values have been 

10 generated, the process 100 returns to the block 116. If, at the decision block 124, all the red, 
green, and blue values have been generated, at a block 126, an image is generated using the 
generated red, green, and blue values. The generated image is then sent to the display 
device 48 for presentation. It will be appreciated that the generated red, green, and blue pixel 
values are applied to corresponding pixels on the display device 48 as determined by a 

15 display processor (e.g., processor 46). 

FIGURE 4 illustrates exemplary processes as performed at the blocks 116 and 122 
(FIGURE 3). Values from an array of thermal data 180 are combined with data from an array 
of pan data 190 to generate red, green, and blue values that are then used for generating an 
image. A value n from a unit within the array of thermal data 1 80 is retrieved and added to a 

20 value p from a unit within the array of pan data 190 at the same location to produce the red 
value. At a block 192, all the values within the array of thermal data 180 are used to 
determine an average or mean for the array of thermal data 180. The generated average or 
mean is added to a value p from a Unit within the array of pan data 190 to generate the green 
and blue values for that location associated with the unit within the array of pan data 190. 

25 FIGURE 5 illustrates an example image 200 produced by the process 100 shown in 

FIGURE 3. FIGURE 5 is an image of the same land area as shown in FIGURE 1 . According 
to the present invention, the level of detail of the resulting image in FIGURE 5 
advantageously is much more detailed than the prior art image generated from thermal band 
data (FIGURE 1). While images shown in FIGURES 1 and 5 are black and white images, 

30 each are created with color for a color display. Items in the image 200 that are red indicate an 
area of the earth that the thermal band data indicates as hot (i.e., above a predefined average 
temperature value). Blue indicates cold (i.e., below a predefined average temperature value) 
and white indicates a mean or average temperature. It will be appreciated that one can alter 
the algorithms of the present invention depending upon the type of thermal imaging the user 

35 desires. For example, if one desires to show blue as hot and red as cold, then one can 
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alternate how the thermal band data and pan band data are used to generate colors on the 
display (FIGURE 4). Also, the images may be manipulated by manually manipulating the 
luminance and intensity levels. It will be appreciated that other color bands may be used. 

It will be appreciated that the pan band data can be corrected or sharpened by any of a 
variety of algorithms. For example, EQUATION (3) below illustrates an exemplary formula 
for correcting the pan band power. 



Pan Corrected Power = 



p pan 
(p2 + p3 + p4)x 2 



(p2 + p3 + p4 )x 2 - p3 x 



g3v g2v 



- p4 x 



_g3v_ 
w4v 



-p2 



w3v 
glv 



w3v 



g2v 



w2v w2v 



+ 



- pi x 



glv 
wlv 



10 (3) 
Where 

pi = multispectral power in band 1 (blue); 
p2 = multispectral power in band 2 (green); 
p3 = multispectral power in band 3 (red); 
15 p4 = multispectral power in band 4 (NIR); 

p pan = multispectral power in band 8 (panchromatic); 

wlv = difference between the wavelength lower limit for multispectral band 1 and 
the wavelength upper limit for multispectral band 1 ; 

w2v = difference between the wavelength lower limit for multispectral band 2 and 
20 the wavelength upper limit for multispectral band 2; 

w3v = difference between the wavelength lower limit for multispectral band 3 and 
the wavelength upper limit for multispectral band 3; 

w4v = difference between the wavelength lower limit for multispectral band 4 and 
the wavelength upper limit for multispectral band 4; 
25 glv = difference between the wavelength upper multispectral band 2 and the 

wavelength lower multispectral band 1 ; 

g2v = difference between the wavelength upper multispectral band 3 and the 
wavelength lower multispectral band 2; and 

g3v = difference between the wavelength upper multispectral band 4 and the 
30 wavelength lower multispectral band 3. 
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The pan correction equation (3) subtracts out average power values of the gaps from 
the total pan power. 

It will be appreciated that the present invention can operate in real-time, thereby 
producing sharpened images at video speed. 
5 While the preferred embodiment of the invention has been illustrated and described, 

as noted above, many changes can be made without departing from the spirit and scope of the 
invention. Accordingly, the scope of the invention is not limited by the disclosure of the 
preferred embodiment. Instead, the invention should be determined entirely by reference to 
the claims that follow. 
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